Conventionally, solid-state imaging elements such as CCD (Charge Coupled Device) and CMOS (Complementary Metal-Oxide Semiconductor) have been widely used for the imaging instruments such as video cameras and still cameras; and optical measurement apparatuses such as component inspection apparatus used in FA (Factory Automation) and optical measurement apparatuses such as electronic fiberscope used in ME (Medical Electronics).
In recent years, there are proposed a large number of techniques for obtaining an image having a wide dynamic-range (referred to as “wide-DR image”, hereinafter) of pixel values in comparison with that of optical film photograph using these solid-state imaging elements.
On the other hand, display apparatuses for displaying moving image and still image such as CRT (Cathode Ray Tube) and LCD (Liquid Crystal Display), projection apparatuses such as projector, and various printing apparatuses are not yet widened in their supportable dynamic range of pixel values at present, and have only a limited range of supportable luminance grayscale. Hence the present status is that, even a wide-DE image should have successfully pictured, there are no apparatuses capable of displaying, projecting or printing the image as it is obtained.
Hence there is a need for a technique (referred to as “grayscale compression technique”, hereinafter) by which the dynamic range of pixel values of the wide-DR image is narrowed, or in other words, the luminance grayscale is compressed, so as to produce an image (referred to as “narrow-DR image”, hereinafter) adapted to the dynamic range of the display apparatuses and so forth.
The following paragraphs will explain a conventionally-proposed grayscale compression technique. The grayscale compression technique can simply be achieved by re-assigning the grayscale of pixel values of the wide-DR image so as to be suited to grayscale of a narrower dynamic range supportable by the display apparatuses or the like.
However, as described in the above, a uniform re-assignment of the grayscale of pixel values of the wide-DR image simply to the narrow dynamic-range only results in a reduced luminance variation of the image as a whole, and consequently in conversion into a poorly-looking image with a degraded contrast. There are conventionally proposed some grayscale compression techniques capable of suppressing the loss in contrast. Three grayscale compression techniques ever proposed will be explained below.
A technique which can be exemplified as a first grayscale compression technique relates to an adaptive determination of a grayscale redistribution rule based on a histogram of luminance of an input wide-DR image (more specifically, calculation of a grayscale conversion curve based on a histogram of an input image). The first grayscale compression technique is on the premise that a principal subject in an image has a large ratio of occupational area, and is to determine a grayscale conversion curve so as to assign the grayscale as much as possible to a luminance value at around a peak in the histogram, to thereby suppress lowering in the contrast of at least the principal subject.
It is, however, difficult to obtain a satisfactory result in every circumference only by an effort based on the grayscale assignment. In an exemplary case where an image has a plurality of principal subjects and has a background with a uniform luminance and a relatively wide area (e.g., blue sky), the subjects often fail in obtaining a sufficient grayscale assigned thereto.
A technique which can be exemplified as a second grayscale compression technique relates to an emphasis of high-frequency components of an image either before or after the grayscale conversion. The second grayscale compression technique is to estimate a portion of contrast lost (or supposed to be lost) through the grayscale conversion, and to compensate the lost portion using a high-frequency filter such as for unsharp masking.
The second grayscale compression technique is advantageous in that it does not raise a problem dependent on composition of image unlike the first grayscale compression technique. The high-frequency filter is, however, causative of overshoot at the contour portion of the subject and of noise emphasis at the flat portion, and is therefore understood as being not capable of always ensuring desirable images.
A technique which can be exemplified as a third grayscale compression technique relates to division of a wide-DR image into a low-frequency-component image and a high-frequency-component image, wherein only the low-frequency-component image is subjected to a proper grayscale conversion processing while leaving the high-frequency-component image unmodified, and the both finally added to produce one synthetic image.
Because the high-frequency-component image is left unmodified in the third grayscale conversion technique, lowering in the contrast due to the grayscale conversion is successfully avoidable. The third grayscale conversion technique, however, still suffers from a problem of overshoot at the contour portion of the subject, and noise emphasis at the flat portion similarly to the second grayscale conversion technique, so that there is also proposed a method of solving the problem by using a non-linear filter (e.g., median filter) in the process of division into the low-frequency-component image and high-frequency-component image.
Summarizing now the first to third grayscale compression techniques described in the above, they can be classified into those effecting the grayscale compression through a relatively local processing using neighboring pixels (first and second grayscale compression techniques), and that effecting the grayscale compression using an entire portion or a relatively large area of the image (third grayscale compression technique). The former results in an unnatural image having only the high-frequency component thereof enhanced, and is far from successful in obtaining effective grayscale compression results. The latter is successful in obtaining a more natural image than obtainable by the former and is said to be more effective in the grayscale compression, because it can adjust also components of relatively low frequencies in parallel to the emphasis of the high-frequency-components.
The latter, however, suffers from a problem in that the process therefor needs a large-capacity memory mainly for the delay line or frame memory, so that it is not adaptive to hardware construction. For instance, the third grayscale compression technique needs a spatial filter for dividing a luminance into a plurality of frequency components, wherein it is necessary to incorporate a large amount of delay lines into the circuit in order to allow installation of a large spatial filter, because a non-artificial, effective grayscale compression is available only when a large spatial filter relative to the image is used.
Meanwhile, for an exemplary case where a function for subjecting a wide-DR image to the grayscale compression processing is intended to be installed on the output section of an imaging apparatus such as digital video camera and digital still camera, there is a large need for the function of grayscale compression processing of the digital still camera, for example, to be incorporated into a hardware, because high-speed signal processing is necessary in order to output image signals while ensuring a predetermined frame rate. Even for a digital still camera for photographing still images, for example, there is a demand for high-speed grayscale compression processing, because it is necessary to output a monitored image to a finder in order to determine a composition of the image.
As described in the above, there is a strong demand for the grayscale compression technique which requires only a small memory capacity to be consumed and a light load of calculation, allows easy hardware construction, and ensures a large grayscale compression effect. This sort of grayscale compression technique has, however, not been proposed yet.
Additional problems, as described below, commonly reside in the above-described first to third grayscale compression techniques.
A first problem relates to generation of overshoot in the luminance at the contour portion of the subject in parallel with emphasis of the high-frequency components.
To suppress this, it is necessary to use a relatively large-sized (20×20 pixels), two-dimensional, non-linear filter. The two-dimensional, non-linear filter of this size expected as being realized on the software basis, however, raises a problem in that cost for the calculation will grow extremely high, and that expected as being realized on the hardware basis raises a problem in that the circuit scale will grow large due to necessity of a large volume of delay lines.
A second problem relates to control of the amount of contrast emphasis of high-frequency components in the high-luminance region and low-luminance region. The above-described second and third grayscale compression techniques are common in that the luminance is divided into a low-frequency component and a high-frequency component, and the grayscale compression is effected by enhancing the high-frequency component while keeping the low-frequency component relatively suppressed.
The emphasis of the high-frequency component, however, results in clipping of the luminance at around the maximum luminance and minimum luminance acceptable by a display apparatus or the like, and consequently in loss of detail of the image, so that the grayscale conversion could not be said as being appropriate, and this raises a need for some countermeasure by which the clipping of luminance is avoidable.
Another problem resides in that an excessive emphasis of the contrast results in an image having an unnaturally enhanced contour portion of the subject, even under a condition not causative of clipping of the luminance.